1. Field of the Invention
The present invention concerns playing media content provided from a remote source.
2. Background Information
Modern cellular networks are evolving rapidly. Over the past few years, with the advent of smart mobile devices, a huge increase of data consuming applications, and a manifold increase in the capacity of cellular network bandwidth, users in cellular networks have become extremely data hungry. Cisco predicts cellular data traffic will grow by over eight times in the next four years, with more than two-thirds of it consisting of mobile video. (See, e.g., the article, Cisco, “Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2011-2016,” available at http://goo.gl/reBfY. (February 2012), incorporated herein by reference.) Traffic from next year's video alone is projected to exceed current total mobile traffic. Moreover, video streaming services are expected to constitute a major portion of the mobile video traffic. (See, e.g., the article, J. Erman, A. Gerber, K. K. Ramakrishnan, S. Sen, and O. Spatscheck, “Over The Top Video: the Gorilla in Cellular Networks,” Proceedings of the 2011 ACM SIGCOMM conference on Internet Measurement, IMC'11, pages 127-136 (ACM, November 2011), incorporated herein by reference.) Indeed, video traffic has increased to the point that it now constitutes more than half of all Internet traffic. (Recall, e.g., the article, Cisco, “Cisco Visual Networking Index Global Mobile Data Traffic Forecast Update, 2011-2016,” available at http://goo.gl/reBfY. (February 2012), incorporated herein by reference.) Together, a wide range of video delivery services and a surge in the quality of videos account for this phenomenon. As more devices become mobile, demand by users for video delivered to their mobile devices is expected to increase.
The data rate available to a mobile user in a cellular network often varies. For example, a user located closer to a base station may experience a higher data rate than one who is far away, at the edge of the macrocell. Further, high data rate small cells such as picocells (deployed by the service providers), femtocells, and potentially even WiFi hotspots (deployed by users) are often overlaid on cellular networks. This results in extreme variations in data rates experienced by a mobile user in a cellular network. Yet another source of bandwidth variability are periods and/or areas of congestion in the network (e.g., during peak hours). Note that network congestion is not limited to wireless networks, and is often found in other communications networks (e.g., “wired” communications networks) as well.
In a video streaming system, the data rate available to a user affects, almost instantaneously, the quality of video experienced by the user. For example, when a user is watching a streaming video, the video quality becomes poor almost as soon as the user moves into a low data rate area. Thus, it will become increasingly important to providing good video streaming services in cellular networks in the near future.
In recent years, there have been industry proposals to exploit adaptive video streaming in wireless networks. With adaptive video streaming, the video bit rate is switched on-the-fly to provide the best video quality to the user based on the available resources in the network. For example, Microsoft's IIS Smooth Streaming (See, e.g., the reference, A. Zambelli, Mobile Video Transmission Using Scalable Video Coding, (Microsoft Corporation, March 2009), incorporated herein by reference.), Adobe's Flash Dynamic Streaming (See, e.g., the article, D. Hassoun, “Dynamic Streaming in Flash Media Server 3.5-Part 1: Overview of the New Capabilities,” http://goo.gl/0G95h (August 2010), incorporated herein by reference.), and Apple's HTTP Adaptive Bit-rate Streaming (See, e.g., the article, R. Pantos (Ed.) and W. May, “HTTP Live Streaming,” (Apple Inc., September 2012), incorporated herein by reference.) use various techniques to deliver streaming video to users efficiently by dynamically switching among different streams of varying quality and bit-rate to provide a smooth and seamless video to users.
The research community has also been very active in this area. For example, an intelligent bit-rate switching based adaptive video streaming (ISAVS) algorithm has been proposed. (See, e.g., the article, X. Qiu, H. Liu, D. Li, S. Zhang, D. Ghosal, and B. Mukherjee, “Optimizing HTTP-based Adaptive Video Streaming for Wireless Access Networks, 3rd IEEE International Conference on Broadband Network and Multimedia Technology (IC-BNMT), 2010, pages 838-845 (October 2010), incorporated herein by reference.) The ISAVS algorithm provides the best possible video quality to users with minimum replay interruptions. Similarly, an optimized H.264/AVC-based bit stream switching for mobile video streaming has been proposed. (See, e.g., the article, T. Stockhammer, G. Liebl, and M. Walter, “Optimized H.264/AVC-Based Bit Stream Switching for Mobile Video Streaming,” EURASIP J. Appl. Signal Process., 2006:127-127 (January 2006), incorporated herein by reference.) The advanced bit stream switching capabilities using SP/SI pictures defined in the H.264/MPEG-4 AVC standard (See, e.g., the article, T. Wiegand, G. Sullivan, G. Bjontegaard, and A. Luthra, “Overview of the H.264/AVC Video Coding Standard,” IEEE Transactions on Circuits and Systems for Video Technology, 13(7):560-576 (July 2003), incorporated herein by reference.) were exploited in the foregoing reference.
Despite resent proposals such as those referenced above, streaming of content, such as video content, still has certain inherent limitations, some of which are discussed in §1.2.1.4 below. Although playing a previously downloaded video generally avoids such problems, downloads are generally much more costly to users than streams.
Traditional Video Delivery
This section discusses traditional video delivery services and introduces their drawbacks when used over modern wireless networks.
Streaming and Downloading
There are presently two ways users can legally consume digital media content in the United States—downloads and streaming. A video streaming service is one where the consumer is not allowed to cache more than a short period of video data ahead of the point being watched. (See, e.g., the reference, “Rates and Terms for use of Musical Works under Compulsory License for Making and Distributing of Physical and Digital Phonerecords,” Title 37 Patents, Trademarks, and Copyrights; Chapter III Copyright Royalty Board, Library of Congress; Subchapter E Rates and Terms for Statutory Licenses; Part 385, 37 C.F.R. §385.11 (February 2009), incorporated herein by reference, which pertains to the analogous concept of audio streaming.) More specifically, according to 37 C.F.R. §385.11, “streaming cache reproduction” means a reproduction of a sound recording of a musical work made on a computer or other receiving device by a service solely for the purpose of permitting an end user who has previously received a stream of such sound recording to play such sound recording again from local storage on such computer or other device rather than by means of a transmission; provided that the user is only able to do so while maintaining a live network connection to the service, and such reproduction is encrypted or otherwise protected consistent with prevailing industry standards to prevent it from being played in any other manner or on any device other than the computer or other device on which it was originally made. Services such as Hulu, Netflix, and Amazon Instant Video are examples of video streaming services. On the other hand, a video downloading service is one where the consumer tries to cache as much of the video as their network bandwidth allows, irrespective of the point of video being watched. Examples of video downloading services include iTunes Movie Rentals, Google Play Movies, as well as YouTube. (See, e.g., the article, C. Breen, “Where to Look for Streaming Movies and TV Shows,” http://goo.gl/JlW0I (July 2012), incorporated herein by reference.) Unlike streaming services in which video playback is aborted when a device loses its connection to the network, it may play all the way to the end in a downloading service. Also, unlike a downloading service, seeking back on the video to replay a portion of it requires the data to be sent again in a streaming service.
Downloads are generally priced much higher than streaming (Recall the definition of “streaming cache reproduction.”) since they confer some ownership rights, as compared with streams, which are consumable only over a limited duration while a “live connection” is maintained. For example, video downloading services are typically ten to a hundred times more expensive than video streaming services, because of the charges imposed by content owners. As a result, from the perspective of price, in most cases, users prefer a streaming service to a downloading service, especially for longer format videos that they are unlikely to view repeatedly.
On the other hand, streaming, especially for content like video, requires a high bandwidth connection (which may be expensive to the consumer and/or the communications service provider) to be maintained over the duration of its consumption. In some cases, such as in a mobile environment or in any access network that delivers variable bandwidth (e.g., DSL or cable), this bandwidth often cannot be guaranteed for the duration of consumption. Traditionally, the problem of maintaining a high bandwidth connection when streaming video has been addressed by either (a) delivering the streamed content at a relatively low bandwidth, at the cost of reducing the quality and aesthetic enjoyment of the media (e.g., low definition video instead of high definition video), or (ii) by adapting the coding rate, and consequently the quality, in real time to match the bandwidth available. Clearly, the latter option also leads to variable quality.
The challenge and expense of maintaining a high bandwidth connection is almost always higher in a mobile environment.
Streaming in Wireless Networks
With the dramatic increase in the use of mobile devices, more users now intend to watch high quality videos on these devices using wireless network connections such as WiFi or 3G/4G/4G LTE and next generation cellular technologies. These wireless networks inherently provide variable bandwidths to users, especially for those who are mobile. Bandwidths experienced by users in these wireless networks can vary from tens of Mbps to a few Kbps, depending on traffic demand(s) from other user(s), and where the user is located with respect to a base station in case of cellular networks, or with respect to an access point in case of a WiFi hotspot. Since higher quality videos require higher data rates, if the user moves to a low data rate region, or if there is traffic congestion, there will likely be insufficient bandwidth to support the streaming of high definition video. In such scenarios, it is expected that video streaming service providers will prefer to lower the quality of the video delivered, rather than causing an interruption in its playback. As discussed, congestion in any type of communications networks (including “wired” networks) can cause similar challenges.
Scalable Video Coding
Lowering the video quality by reducing its bit rate can also be implemented using SVC (i.e., scalable video coding), an extension of the H.264 video coding standard. SVC allows a high quality video to be decomposed into multiple bit streams, with a subset of these bit streams requiring a lower bandwidth that can be used to display a lower quality version of the original video. In other words, a video can be divided into several bit stream layers such that each additional upper layer adds to the quality of the video. Further, every layer consists of predictions based on data decoded by (e.g., typically all of) the layer(s) below it. Thus, every layer directly or indirectly depends on its lower layer(s), and can only be used when (e.g., typically all) layer(s) below it are available to be decoded. The lowest layer, referred to as the “Base Layer” of the video, can be decoded by itself, independent of any other layer. The higher layers of the video that progressively enhance its quality are referred to as “Enhancement Layers” of the video.
Given its scalability in quality and bit rate of the video, SVC is considered to be a suitable encoding method for mobile TV broadcast/multicast (See, e.g., the article, S. Hua, Y. Guo, Y. Liu, H. Liu, and S. Panwar, “Scalable Video Multicast in Hybrid 3G/Ad-Hoc Networks,” IEEE Transactions on Multimedia, 13(2):402-413 (April 2011), incorporated herein by reference.) as well as video streaming services (See, e.g., the article, T. Schierl, T. Stockhammer, and T. Wiegand, “Mobile Video Transmission Using Scalable Video Coding,” IEEE Transactions on Circuits and Systems for Video Technology, 17(9):1204-1217 (September 2007), incorporated herein by reference.). The video to be streamed is first divided into chunks, where each chunk contains data for a small temporal portion of the video (e.g., on the order of one second of video). In simpler terms, the video can be represented as the sequential playlist of all its temporal chunks (simply referred to as “chunks”, without loss of generality). Each chunk is then divided into a base layer and one or more enhancement layers using SVC. The chunks are then streamed, in sequence, to a user device. At the user device, the chunks are decoded and played, one by one, as they become available. In general, a chunk cannot be played while it is still being downloaded. Under SVC, the user device tries to download as many layers of a chunk of video as the available bandwidth allows, until it is time to start playing the chunk. Using SVC, user devices can avoid interruptions by continuing to play the video at a lower quality when their bandwidth drops, by downloading fewer layers of the chunks of the video.
Limitations of Streaming
Although using SVC for streaming videos over wireless networks helps to reduce interruptions to the video as the user experiences varying bandwidths, it still suffers from a few drawbacks when compared to other kinds of video delivery services. For instance, since a user device decoding and playing streaming video cannot (legally) cache (e.g., too many) future chunks of the video (even if they are close to the base station or access point, and have surplus bandwidth available), the quality of video drops as soon as such user devices move away from the base station or access point (and/or as soon as congestion occurs) and their bandwidth falls below the required level to download all layers (that is, the base layer and all enhancement layers) of the video.
On the other hand, when a use device downloads (as opposed to plays a stream of) a video, the surplus bandwidth available can be used to download future chunks of the video. Thus, with downloading, even when the available bandwidth falls, the user can continue to enjoy the same high quality video since future chunks were stored in cache. Unfortunately, however, as discussed above, downloading video content is generally much more expensive than streaming such content.
Perceived Needs
As should be appreciated from the foregoing, although a user may prefer to use a video streaming service from the price perspective, a video downloading service may be preferable from the quality perspective. Consequently, a service that can potentially provide download quality video, while still qualifying legally as a streaming service, is highly desirable for wireless networks.